The present invention relates to the broad field of electro-optical systems which employ area correlation video trackers, and more particularly, to an adaptive reference threshold update processor for adaptively updating the intensity pattern of a reference array of pixels based on changing video scene conditions of said correlation video tracker.
Recently, the operational requirements imposed on electro-optical guidance assistance systems on-board single seat high performance aircraft, like fighters, for example, have been in a direction to minimize operator interface during acquisition and subsequent tracking of a target. Digital Area Correlation Trackers (DACT) have been used to reduce the pilot's attention and command inputs in accordance with prespecified goals of a "single seat" mission. In these DACT type systems, a visual display, such as a TV monitor, provides the field of view (FOV) scene sighted by an electro-optical sensor, such as a TV scanner, which is mounted on the aircraft. The line-of-sight positioning of the electro-optical sensor is normally controlled by position loops of a gimballed servo system as governed principally by one or more position error signals usually associated with azimuth and elevation. The FOV display is sectioned into at least two window areas by a set of cursors, for example, one window area generally referred to as the Track window and another window area known as the Reference window being of a smaller area than the Track window and positioned in the visual display within the Track window (refer to FIG. 1).
In one embodiment of operation of a DACT system, a reference image representative of a predetermined target or selected portion thereof is provided in the Reference window of the display. The reference image comprises an array of digitized video picture elements known more commonly as pixels. The reference array of pixels represents a desired intensity or gray level pattern of the selected image for tracking purposes. At acquisition, the reference array of pixels which is defined by the geometric position of the Reference window within the Track window is extracted from the acquisition TV frame of video information and stored for cross-correlation purposes with video information of subsequent TV frames. FIG. 1 exemplifies the position of the Reference window 10' about the crosshairs 12' within the Track window 14' of the field of view (FOV) scene during an acquisition TV frame 1a.
The stored reference array acquired from the initial or acquisition TV frame is automatically correlated by the DACT with picture data within the Track window which is received in subsequent TV frames from the TV sensor. Each cross-correlation step is generally one of image matching where the area array reference of pixels, which may be a 16.times.16 array, for example, is image compared with each possible 16.times.16 array zone of pixels in the Track window of a subsequent TV picture frame in accordance with some known metric function. For example, suppose that the Track window pixel array area is 32.times.32 pixels, then there exist 289 possible 16.times.16 pixel array overlay image matching comparisons within the Track window of each TV frame subsequent the acquisition TV frame. As a result of the cross-correlation operation carried out for each subsequent TV frame of picture data, the Reference window is repositioned in the zone of the Track window which yields the best image match. An exemplary depiction of the Reference window 10' being repositioned in the Track window 14' in a subsequent TV picture frame 1b is shown in FIG. 1. In most DACT's, the relationship of the position of the best match reference image zone with respect to the position of the acquisition reference image zone provides a measure of the servo position tracking errors which are generally used to govern the gimbal servo system controlling the position of the electro-optical sensor to automatically track the selected target image.
At some time after acquisition, the reference array of pixels, in most cases, will no longer have the appearance characteristics of the selected target image as viewed by the TV camera because of practical conditions such as changing range of target, aircraft maneuvering, change in aspect angle and lighting, for example. If the new target patterns are permitted to change to the extent to cause loss of track, the operator's or pilot's attention may be further burdened by having to repeat an acquisition procedure with each loss of track occasion. In keeping with the goal of reducing the operator's attentiveness with respect to manually operating the DACT, it is of significant importance to automatically update the reference array of pixels at times during the tracking operation to prevent, for the most part, loss of track which is undesirable because it requires more pilot attention to repeat the acquisition procedure for each occasion thereof.
One known processor for updating the reference array during the tracking operation of the DACT employs one or more predetermined threshold values which represent reference array intensity pattern change limits for a set of known video conditions. For a more detailed description of such a system, reference is made to U.S. Pat. No. 3,955,046 issued to Ingham et al on May 4, 1976. In this type of processor, the intensity pattern of the reference array is correlated with the video information of the Track window for each subsequent frame to derive a measure of the change in the reference array intensity pattern with respect to time. The metric used to measure change in intensity pattern is usually some known image matching function. A selected measured change value for each TV frame is compared with one or more selected predetermined threshold values to determine in which TV frame to update the reference array of pixels. Generally, these types of systems are adequate if the changing video conditions are known for given target images and the intensity pattern of the video scenes is changing very slowly. However, when dealing with a high performance aircraft, like a fighter, for example, which may at times perform such maneuvers as dives, turns, loops, jinks and even pop-ups with respect to a target, the intensity pattern of the video TV frames is changing dynamically, varying widely at relatively high speeds. Under these undesirable dynamic video conditions, it is almost impossible to generate enough threshold values to suit all the changing video conditions. For this reason, it appears that this type of updating processor is not suitable for the situations described above and more than likely track will be lost in those instances.
Another known processor for updating is one which updates the reference array periodically (i.e., the reference array is rederived automatically every W frames, W being arbitrary or preset). However, experimentation of these systems has demonstrated that, in some cases, each time the reference array is updated, a slight difference in the intensity pattern occurs, resulting in the Reference window walking through the video scene in the subsequent TV frames. This phenomenon is more commonly referred to as "random walk".
Still another known processor for updating is one which updates the reference array using a recursive update procedure. That is, each time a new reference array of pixels is acquired, a weighted average or integration is performed based on a past history of reference arrays to derive the updated reference array for use in cross-correlation of subsequent TV video frame information. Here again, in those cases in which the aspect angle of the image may be constantly changing as a result of aircraft maneuvering, for example, the averaged periodic updates of the reference array of pixels may generate a new image which is of some form other than the originally perceived target image portion. For example, if the originally acquired target image in the Reference window has the form of a rectangle and for some reason, this rectangular image undergoes a rotation with time due possibly to aircraft maneuvering, then the resultant averaged reference array image after a number of updates may resemble a circle or some other form closely associated therewith. This problem appears similar to the problem of the periodic updating processor. Consequently, it is likewise expected to cause a "random walk" phenomenon in some instances.
Apparently, a fundamental problem with the updating processors of the DACT's is to know when to automatically update the reference image or array of pixels. Updating the reference array is generally necessary when the perceived target or selected portion thereof within the Track window changes in size or aspect angle or when the intensity pattern or structure of the image is caused to change. The known updating processors described hereabove appear to be inadequate in all cases for use in video tracking because they require undesirable attentiveness, in some cases, in reacquiring a new reference image once track is lost. A more desirable reference image updating system may be one which can adaptively update the reference array of pixels in accordance with the actual dynamically changing video conditions so as to minimize attentiveness with regard to the operation of the DACT system.